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Introduction 


The European Association for Quality Assurance in Higher Education (ENQA), in 
cooperation with the Quality Assurance Agency (QAA, UK), organised a seminar on 
theme ‘Internal Quality Assurance - Enhancing quality culture’ which was held on 
8-9 June, 2010 in London, United Kingdom. The seminar marked the fourth annual 
meeting of the ENQA IQA Group with the ENQA IQA Steering Group being the main 
responsible for planning and organising the event. 

The purpose of this seminar was to unite quality assurance (QA) agencies and other 
stakeholders that are interested in operational internal quality assurance (IQA) systems, 
and/or who are planning to implement such a system. The seminar provided a platform 
for the staff of QA agencies for discussing shared challenges, concerns, innovative ideas 
and future expectations which may then be applied to individual IQA practices and/or 
future IQA processes. 

This report collects some good practises that are implemented for internal quality 
assurance in different QA agencies, with the intention to share them for the benefit of 
other quality assurance agencies or institutions interested in these issues. In addition 
to good practices, the participating agencies were encouraged to openly share what 
processes they find challenging or ineffective in their agencies. This report thus 
provides a short list of both good and ineffective practices as they were presented in the 
seminar in London. 
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Quality Assurance in Commission des 
Titres d’Ingenieur (CTI) 

Teresa Sanchez Chaparro 


About CTI : 

Commission des Titres d’lngenieur (CTI) is an independent body involved in the 
development of the European Higher Education Area, established by the Erench law in 
1934. 

Its missions are respectively: 

• The evaluation and accreditation of higher education institutions in the fields of 
engineering, computer science, applied mathematics, project management, etc; 

• the development of quality in engineering education; 

• the promotion of engineering curricula and careers in Erance and abroad (e.g. 
Germany, Switzerland, Bulgaria, Vietnam). 

Source : http://www.cti-commission.fr/ 


Recent issues and organisational changes 

hollowing its external review of 2009 carried out for ENQA membership purposes, 

CTI has introduced a number of organisational changes in order to introduce the 
recommended improvements in two main domains: 

1. The reinforcement of its organisational structure; 

2. the development of information tools in order to facilitate the traceability of its 
evaluation and accreditation process (evaluation and accreditation of engineering 
programmes in Erance). 

Up to 2009, CTTs administrative functioning was based in a non permanent structure 
composed by people belonging to different organisations and partners, which lead 
to coordination difficulties. Two persons have been recruited in order to create a 
permanent team: a person in charge of coordinating all administration activities, and a 
programme director, with tasks related to internal quality assurance and international 
relations. 

Quality assurance issues are addressed through ad-hoc projects, coordinated by the 
programme director and supervised by one of CTTs member organisations. The first 
quality assurance project launched under the new structure is the development of 
an Internet-accessible internal information system (MySQL database). This has been 
created to improve process traceability. 

This project is a continuation of an initiative launched in 2008. A first pilot was built 
at that time in collaboration with the Conference of Directors of Erench Engineering 
Graduated Schools (CDEEI). The idea was to develop CTTs database using CDEETs 
development capabilities. A developer from CDEEI was assigned to the project and a 
specific Internet development environment was taken into use, which was already in 
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use at CDEFI. The collaboration between CTI and CDEFI offered potential additional 
benefits, such as the possibility of establishing a common database between the two 
organisations (with some basic information). 

At this stage of the project, the structure of the database has been redefined in 
order to assure an efficient and effective organisation of information. A set of key 
strategic issues regarding confidentiality and property of information and access rights 
have been identified and clarified. The development phase began in February 2010. A 
number of difficulties have come up since the start, coming from different origins, such 
as a challenge in the sharing of human resources between two different organisations, 
technical rigidities of the chosen development platform and difficulties due to the 
coexistence of two different data structures in the same database. CTI is currently 
trying to address these challenges in order to get the best benefits from the project. 


DOCUWARE - ACSUCYL’s 
Document Management System 

Sandra Marcos Ortega 


About ACSUCYL: 

ACSUCYF is the external assessment body for the university system in Castilla y Feon 
in Spain and is aimed at the assessment, accreditation and certification of quality at 
Universities and research and higher education institutions in Castilla y Feon. 

In addition, ACSUCYF can also collaborate in the processes of assessment, 
accreditation and certification of universities and higher education institutions outside 
the Autonomous Region of Castilla y Feon in the context of the European Higher 
Education Area. 

The activities of assessment, accreditation and certification developed by the agency, 
aim at achieving the following general purposes: 

a. To foster the improvement of teaching and research activity and management 
of Universities and research and higher education institutions, encouraging the 
improvement of competitiveness and economic development of Castilla y Feon. 

b. To provide adequate information about the university system to public 
Administrations, the productive sector and society in general, for making 
decisions in their areas of action. 

ACSUCYF undertakes the functions which have been legally assigned to it in an 
autonomous manner and in accordance with European standards for quality assurance 
agencies. 

Source: http://www.acsucyl.com/ 


In order to enhance and speed up internal handling of documents, ACSUCYL uses 
an automatic document management system. This application, called DOCUWARE, 
covers all of the documents involved in internal quality management and information 
security management systems. 

Documents included in the application are grouped into various files depending on 
the area to which they belong. There are four different files: 

1. SGC documentation: Contains all the processes and procedures relating to 
the activities conducted by the agency as well as any documents which affect or 
might affect its running, such as legislation, rules governing internal functioning, 
European criteria etc. 

2. SGC registers: Contain evidence of compliance with quality criteria. Amongst 
other things, these files contain a record of the planning behind each assessment 
process, follow-up indicators, non-compliances within the system, corrective and 
preventive measures to be implemented, staff training files etc. 

3. SGSI documentation: Contains all documents related to the information security 
management system in place at ACSUCYL. 

4. SGSI registers: Contain evidence of compliance with ISO 27001 regarding 
information security systems, such as a record of all outgoing material, follow-up 
indicators, non-compliance within the system, corrective and preventive measures 
to be implemented in relation to information systems etc. 

DOCUWARE also provides a common tray which all ACSUCYL employees have access 
to and which they can use to share information. In addition, each employee has their 
own personal tray within the application through which they can manage their own 
documents. 

The main advantages of a document management system are: 

• Easy access to well organised and structured information. 

• A common repository where all the documents relating to internal quality 
assurance systems can be stored. 

• Control over the versions of documents, thereby ensuring that all employees are 
working with the latest version of any document. 

• System for digital signatures, thereby facilitating and streamlining internal 
handling of documents. 

• Centralised handling for planning assessment and follow-up processes. 

DOCUWARE is managed through authorisation which is allocated by the system 
administrator, thus ensuring that access to the information and documents is 
controlled. In general, all employees have access to documents describing how 
ACSUCYL conducts its activities and how the agency works. The application contains a 
powerful search engine enabling fast and easy access to information. 

In order to use this application in their everyday work, the ACSUCYL staff are 
trained in its use and handling. The staff are given regular refresher courses in the use 
and functions of the application, thereby enabling them to obtain the maximum benefit 
from it. 
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Since March 2010, the document management system has been functioning via 
Internet, thereby providing a more user-friendly environment for access to information, 
as well as the access to the supplier management database. 
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Good and Ineffective Practices in IQA 
at ACSUG 


Luis Carlos Velon Sixto 


About ACSUG: 

The Axencia para a Calidade do Sistema Universitario de Galicia (Agency for Quality 
Assurance in the Galician University System, AGSUG) was legally established on 
30 January 2001 as a consortium between the Regional Government of Galicia and 
the three Galician universities. ACSUG has full legal personality and the necessary 
independence to be able to achieve its objectives to rigorous standards fully respecting 
university autonomy. 

ACSUG was founded in order for the university system in Galicia to attain high levels 
of prestige and recognition among Spanish, European and international universities. 




The agency was established based on two points of view. The first is the principle 
that the new financing programme will take the quality of universities into account, 
allocating part of the resources to the improvement of teaching, management, etc., 
and the second arises from previous experiences in the field of assessment of new 
international trends and mainly from the recommendations by the Council of Europe 
on European co-operation in order to ensure quality in higher education. 

The aim is for Galician universities to be distinguished firstly by the development 
of their autonomy, responding to the specific socio-economic and cultural needs that 
arise in the immediate context of the Autonomous Region of Galicia and secondly, by 
their competitive integration in the Spanish and international university context. These 
two points can be summarised as autonomy in order to promote their distinguishing 
features, and integration in order to promote innovation and development. 

AGSUG creates a framework for co-operation and co-ordination between the 
government, the Galician universities and other higher education and quality 
assessment bodies, both in Spain and abroad, to exchange opinions, debate and join 
forces. This will undoubtedly lead to the improvement of quality and the prestige of our 
university system. 

Source: http://www.acsug.es/ 


1. Introduction 

It is vital that the channels of internal and external communication within agencies 
are well defined and effective. Information has to be communicated to the staff in due 
time. Achieving this will prevent problems of coordination and help to organise projects 
better. Other possible benefits are the improvement of working environment and a 
greater involvement of the staff. 

2. Internal communication in ACSUG 

Good practices: 

• Periodic staff meetings: Organisation of the meetings sufficiently in advance, 
agenda for issues to talk about, discussion item of general information about the 
current activities, an item for improvement proposals from the staff. 

• Tools to share information: Internal information network; databases of legislation, 
suppliers, agreements etc; shared folders divided in different activities; calendar 
with the activities of each day. 

• It is important to clearly appoint a person responsible for internal communication 
and for fostering the general participation of all staff. 

Ineffective practices: 

• Agency directors and coordinators have to promote and emphasise the importance 
of internal communication. 

• The information interchange has to occur in all directions: not only in top-down 
direction (which is a common direction). At the same job level, transversal and 
down-top communication are necessary. All staff can have good ideas to improve 
the agency’s activities: “Several little improvements can cause a great change”. 
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3. External communication in ACSUG 

Good practices: 

• Satisfaction surveys on the activities: It is important to insist in the relevance of 
participation in surveys. 

• Information meetings with university stakeholders, not only with rectors or 
people with high level responsibilities. 

• Computer applications provide a fast tool to manage requests, to include 
information and share it, and to allow remote access to agencies’ websites/ 
institutions’ databases. 

• Website: It is very important to have a well organised and updated website, 
because most people have contact with the agency only through the website. 

Ineffective practices: 

• Suggestion box on the website and in the main office: The main idea is good but 
people usually prefer to make phone calls for suggestions. 

• Computer applications: People may have insufficient computer skills, and 
legislation may limit the level of computerisation implementation. 

• Website: As it is said above, if you do not dedicate the necessary time to update 
the website, the general opinion of the users will be not very good. 
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IQA at AEQES 

Caty Duykaerts 


About AEQES: 

AEQES is funded by the Ministry of the French Community of Belgium. It is 
independent of both the institutions of higher education and the government of the 
French Community. 

AEQES is responsible for the external evaluation of the quality of higher education 
in the French Community in Belgium. The agency evaluates universities, “hautes 
ecoles” (non-university HEIs), art academies and conservatoires, higher institutes of 
architecture and adult learning institutions. 

AEQES is the only agency officially recognised in the French Community. It aims at 
stimulating co-operation among HEIs, developing quality culture and disseminating 
good practices. It is in charge of the policies and guidelines concerning the external 
evaluation of HEIs and their programmes in the area. 

Source : www.enqa.eu 


1. Assessment of programme clusters 

The main objective of assessing programme clusters is to get an overview of similar (or 
almost similar) programmes in the process of a peer-review evaluation conducted by 
one panel. The challenge is to compose such a panel (different profiles of peers with 
a wide scope of disciplines to deal with); and for the HEIs, to write a coherent self 
evaluation report, integrating sometimes different faculties. 

AEQES has recently worked with a large cluster in sociology at a university level, and 
will soon work in a cluster of computer science programmes offered by universities and 
other HEIs. 

In implementing its evaluation plans, AEQES tries to consider the double perspective 
of curricula logic and labour market reality in terms of employability of the degrees. 

2. Building a framework for experts writing their reports 

One major concern is how to build a framework that provides sufficient freedom 
to experts writing their reports and meets the minimum standardisation of layout 
for publication. For the time being, experts are asked to write their reports with 
the support of the agency. However, this may not be the best practice, given that 
different panels may produce different results. In fact, AEQES has just experienced the 
production of reports by four different panels with quite different results. 

3. Selecting the right chair for a panel of experts 

Presently, the selection of the chair to a panel of experts is made by the agency on the 
content of the CVs and other documents completed by the potential experts: academic 
background, professional experience, QA competencies, previous experience in peer- 
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reviews, etc. However, the satisfactory choice of a chair becomes visible only during 
the site visits and the production of the reports. The main problems that the chair 
may have are weak leadership or poor listening skills, or both. This may lead in the 
alteration of group dynamics. 

4. Follow-up procedures 

There are two aspects to consider: 

• Debriefing with HEIs 

When an external evaluation phase is over (including the publication of the 
individual reports, the global analysis and the follow-up strategic plan), HEIs are 
requested to answer a questionnaire about how the whole process was conducted. 
The results are analysed by the AEQES team and the institutions are invited to 
take part in a debriefing meeting to analyse the survey feedback and to discuss 
the various points. This is an important opportunity to collect perceptions on our 
practices, reflect on them and understand the institutions expectations better. In 
other words, it is definitely a good moment for communication. 

• Eollow-up procedure for each institution 

Our agency is responsible for the formative assessment of study programmes, 
there’s no accreditation and no formal consequence for HEIs. This quality 
enhancement perspective is balanced by the publication of the report 
(accountability perspective) and follow-up strategic plan of each institution. 
Within 6 months after the publication of its report, each institution sends the 
agency a follow-up strategic plan that is posted online in direct link with the 
report. The institution is allowed to update this strategic plan every three years. 
This legal procedure is described in the 2008 Act and since we are presently in 
the first implementation of it, it is too early to draw conclusions. 
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Internal Quality Assurance of 
Reviewer Selection and Monitoring at 


AKKORK 


Yuri Rubin and Erika Soboleva 


About AKKORK: 

AKKORK is an independent professional agency in the field of consultancy, conduct 
of the reviews, accreditation and certification of the education institutions. 

The key fields of activity: 

• Consultancy on the issues of quality improvement and education quality 
assurance at the institutional and program levels; 

• Consultancy on the issues of management quality improvement of the education 
institution; 

• Education quality and quality assurance review; 

• Independent accreditation and certification; 

• accreditation of education programs and institutions at the institutional and 
program levels; 

• Legal consulting; 

• Financial activity effectiveness review of education institutions. 

We are seeking to contribute to the development of education institutions and the 
whole education system and we will help you in optimization of management and 
internal control systems, as well as in discovering your education institution potential 
and competitiveness. 

Independency and impartiality of AKKORK, as well as competence of our experts 
are corroborated by the accreditations in the leading international accreditation 
networks of education quality assurance and by the positive experience in the period 
of 6 years. 

Source: http://www.akkork.ru/e/ 


The Agency for Higher Education Quality Assurance and Career Development 
(AKKORK) uses the following internal quality assurance system in reviewer selection 
and the monitoring of their job performance: 

1. The recruitment of reviewers consists of the following elements: 

- Selection of reviewers 

- Assessment of certain experts’ adequacy to work at the agency 

- Training of reviewers 

- Certification of reviewers 
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2. The specialisation of reviewers. Reviewers specialise in areas of education or 
quality assurance. They are certified for these tasks and work only in the activities 
that they are certified for. 

3. The monitoring of reviewers. The reviewers are organised in groups based on 
their specialisation. Each group has its own supervisor. These supervisors are 
experts both in the area of the reviewers’ specialisation and in human resource 
management. The monitoring of reviewers includes the following elements: 

- Unofficial interviews of university clients are carried out; 

- Evaluation of the quality, professionalism and fairness of reviewer reports; 

- Reviewer files are placed on the agency’s website. 
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